Suchergebnisse

Sound questionnaire translation is crucial for collecting equivalent data in cross-national surveys. The topic is widely discussed in fields in which translated questionnaires are used, such as in the social sciences or in cross-cultural psychology, but hardly in translation studies. This article aims at bridging this gap by bringing the genre of questionnaires closer to translation scholars and practitioners. To begin with, we will provide a sketch of key characteristics of cross-national survey methodology, with a special focus on questionnaire translation. Next, the multi-dimensional concept of translation competence will be introduced. Along the ISO 17100 competence model then, we will list requirements, challenges, and resources for questionnaire translation and illustrate those with examples. The concluding paragraph will highlight areas for interdisciplinary research involving both translation studies and cross-national survey methodology.

Back translation - the 're-translation' of a translated questionnaire back into the original language and the subsequent comparison of the original version and the back translation - is a controversial assessment method for questionnaire translations. Recently, cross-cultural survey methodologists have followed the call for more empirical research on this method. This article adds to the small body of research by drawing on the back translation documentation from the 2012 European Quality of Life Survey (EQLS). First, results from the official EQLS back translation step are contrasted with results when additional persons perform the same comparison step between back translation and original. We note inconsistency in the detection of presumed flaws. Second, the back translation outcome is contrasted with additional native speaker checks of the actual translation. While back translation can uncover problems, it causes quite a number of false alarms, and even more importantly, many problems remain hidden.

Open-ended probing questions in cross-cultural surveys help uncover equivalence problems in cross-cultural survey research. For languages that a project team does not understand, probe answers need to be translated into a common project language. This article presents a case study on translating open-ended, that is, narrative answers. It describes how the translation can be approached to obtain high-quality translations and how translation, coding of answers, and analysis of codes are all deeply interwoven. While the translation of narrative answers is a small field of application, the article's content is also relevant for those who are increasingly setting up cross-cultural cognitive interviewing or for qualitative researchers in general who pay increased attention to the translation of qualitative research data.

AbstractWhen it comes to quality in questionnaire translation and hence comparability in comparative research, the chosen translation method is crucial for the outcome. Few empirical studies compare different translation methods—a fact which is often deplored in the research community. To fill the gap, in this study, the team translation approach is compared against a simple back-translation approach. The starting point in both cases was the initial English–German translations of ISSP (International Social Survey Program) questions. The final translations from both approaches were assessed, with a focus on how translation issues, such as mistranslations or wording issues identified in the initial translations were addressed. While none of the twenty-nine issues in the initial translation were present in the final team translation version, twenty-two of these issues were still present in the final version after the back-translation approach. For a selected number of items, we also ran a split-ballot experiment in a web survey. Only five out of fifteen items (33 percent) that went into the experiment showed significant differences between the translations, and only one could clearly be attributed to remaining errors in the back-translation version. In sum, the final translation from the team approach clearly outperformed the final translation from the back-translation approach when it comes to text-based criteria (in particular, accuracy and fluency). The quantitative test showed that many translation issues (those remaining in the translation after the back translation step) had no effect on the estimates. Nevertheless, we ask respondents to put effort into survey responding; in the same vein, we as researchers should put effort in the survey experience by providing questions that are clearly worded and free of errors, which puts the team approach ahead of the back-translation approach.

This report summarizes the main themes and conclusions from the OECD-GESIS Seminar on Translating and Adapting Instruments in Large-Scale Assessments, which took place at the Organization for Economic Co-operation and Development (OECD), Paris, in June 2018. The five sessions covered the topics (1) etic (universal) vs. emic (culture-specific) measurement instruments, (2) language- and culture-sensitive development of measurement instruments, (3) international guidelines vs. implementation in countries and by translators, (4) tools and technological developments, and (5) quality control of translations. Key players in the field presented on best practice, lessons learned, and innovations and also made suggestions for moving the field forward.

"Good questionnaire design and high quality questionnaire translations are vital for data comparability in cross-national survey research. With the aim to ensure comparability and prevent unintended deviations, major academically-driven studies such as the International Social Survey Programme (ISSP) or the European Social Survey (ESS) annotate the source questionnaire specifically
for translation, thus providing guidance on what needs to be considered in translation. This paper studies these translation annotations, a topic having received scant attention in research so far. The goal of this paper is to raise awareness on this special support structure in comparative
research as well as on potential pitfalls in questionnaire translation. To this end, first,
translation annotations, mainly from the ESS and the ISSP, are analyzed with a view to setting up a classification of translation annotations. Second, examples of annotation types are presented together with what questhey meant for translation. Third, merits of annotations and potential criticism are discussed. Fourth, guidelines are ventured for writing translation annotations as well as for working with them. Fifth, research fields are listed in order to further explore
the issue of translation annotations and its impact on translation and comparability." (author's abstract)

This chapter examines the technical challenges involved in translating and adapting measurement instruments, i.e., questionnaires, for migration research. The first part outlines good practices in questionnaire translation. In line with the technology-based focus of this book, the second part focuses on computerized surveys and on the interplay between technology, language, and culture. Frameworks from the software localization field are consulted and transferred to the context of computerized multilingual surveys with respect to their impact on source questionnaire design and on translation and adaptation. Real-life examples come from our own experiences in international and migration research, as well as from a review of existing reports and research articles. The main goal of this chapter is to raise awareness of the additional technology layer that impacts translation and adaptation, with an ultimate goal to improve translation and adaptation processes, and the outcomes of migration research.

Methodological studies usually gauge response quality in narrative open-ended questions with the proportion of nonresponse, response length, response time, and number of themes mentioned by respondents. However, not all of these indicators may be comparable and appropriate for evaluating open-ended questions in a cross-national context. This study assesses the cross-national appropriateness of these indicators and their potential bias. For the analysis, we use data from two web surveys conducted in May 2014 with 2,685 respondents and in June 2014 with 2,689 respondents and compare responses from Germany, Great Britain, the United States, Mexico, and Spain. We assess open-ended responses for a variety of topics (e.g., national identity, gender attitudes, and citizenship) with these indicators and evaluate whether they arrive at similar or contradictory conclusions about response quality. We find that all indicators are potentially biased in a cross-national context due to linguistic and cultural reasons and that the bias differs in prevalence across topics. Therefore, we recommend using multiple indicators as well as items covering a range of topics when evaluating response quality in open-ended questions across countries.

In 2012, a new question was introduced into the International Social Survey Program (ISSP). It asks respondents to indicate what they consider the best division of labor between men and women. In this paper, we propose to assess the validity and cross-national comparability of this new ISSP question, using a mixed-methods approach that combines quantitative experimental data with qualitative probing data. We implemented our experiment in non-probability online surveys in five countries, in which half of the respondents received the original ISSP question and the other half a variant with an additional category saying "Each family should find the solution which works best for them." In addition, the understanding of "individual solutions" was probed. We report on the understanding of this category.

Due to the growing significance of international studies, the need for tools to assess the equivalence of items in international surveys is pressing. Online probing is a powerful tool for identifying the causes of non-equivalence; it incorporates probing techniques from cognitive interviewing in cross-national web surveys. So far, our online probing approach has applied three different probe types - category-selection probes, specific probes, and comprehension probes - to inquire about different aspects of an item. Previous research has mostly asked one probe type per item but in some situations it might be preferable to test potentially troublesome items with multiple probe types. However, empirical evidence is missing on whether the sequence of probe types has an impact on response quality, respondents' motivation, and answer content. In this study, we report evidence from a web experiment that was conducted with 1,354 respondents from Germany, Great Britain, the U.S., Spain, and Mexico in June 2014. In this experiment, we asked respondents three different probes for one item and we manipulated the sequence of probes in each experimental condition. Our research indicates that the sequence in which different probe types are asked has an impact on response quality, the respondents' motivation, and probe answer content. However, the respondents in the five countries react differently on the variation in the probe sequence suggesting that response behavior on probes is partly culturally driven.

Migration is a global phenomenon; the number of migrants and refugees worldwide has risen rapidly in recent decades - and especially in recent years. These developments come with the need to better understand the living conditions of migrants and refugees and the processes of integration in order to ensure peaceful and sustainable integration in societies. Surveys are an effective means of data collection in this regard. From a methodological point of view, however, a number of challenges are associated with surveys among migrants and refugees. These proceedings, which mainly draw on the contributions to an international symposium, focus on the survey questionnaire, its translation and adaptation as well as on cultural and linguistic challenges encountered during fieldwork. The goal of this publication is to draw attention to the methodological challenges in migration and refugee research in order to foster the collection of valid and reliable data.

The recommendations in this guideline focus on the concrete translation step, answering the
question what those translating questionnaires need to keep in mind and consider when
translating and taking actual translation decisions. Areas covered include: (a) the translation of
meaning, (b) cultural adequacy, (c) response scales, d) consistency, e) style and register, f)
linguistic correctness, and g) layout and presentation. We particularly have questionnaire
translations for cross-cultural or cross-national surveys in mind where comparability is
paramount.

In a small-scale experiment, questionnaire translations (English into German), either produced via a simple back translation approach or via a team translation approach, were implemented in a split-ballot web survey.

The questionnaire contained items from the International Social Suvey Program (ISSP) modules 2019 and 2020.

The overarching goal was to assess the impact that different translations - produced by different methods - have on the resulting survey data.



The data provided here contain the dataset with all closed-ended items (open-ended probes were removed) and the analysis code that was used for the paper:



Behr, D., and Braun, M. (forthcoming). How does back translation fare against team translation? An experimental case study in the language combination English-German. The Journal of Survey Statistics and Methodology.



The paper describes the methodology of the study in detail.

Answers to cognitive probes implemented in (cross-national) web surveys ("web probing")
(1) Replication of closed-ended items on attitudes towards immigrants from the International Social Survey Program (ISSP), addition of a follow-up probe:
Immigrants increase crime rates.
Immigrants are generally good for Germany's economy.
Immigrants take jobs away from people who were born in Germany.
Immigrants improve German society by bringing in new ideas and cultures.
Probe after each item (split): Which type of immigrants were you thinking of when you answered the question?

The study and results are described in detail here:
Braun, Michael, Dorothée Behr, and Lars Kaczmirek. 2013. "Assessing cross-national equivalence of measures of xenophobia: evidence from probing in web surveys." International Journal of Public Opinion Research 25 (3): 383-395. doi: http://dx.doi.org/10.1093/ijpor/eds034.

(2) Replication of item on civil disobedience from the ISSP, addition of a follow-up probe:
How important is it that citizens may engage in acts of civil disobedience when they oppose government actions? (Not at all important 1 ... Very important 7)
Probe: What ideas do you associate with the phrase "civil disobedience"? Please give examples.

The study and results are described in detail here:
Behr, Dorothée, Michael Braun, Lars Kaczmirek, and Wolfgang Bandilla. 2014. "Item comparability in cross-national surveys: results from asking probing questions in cross-national web surveys about attitudes towards civil disobedience." Quality & Quantity 48 (1): 127-148. doi: http://dx.doi.org/10.1007/s11135-012-9754-8.